Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Integrating posterior probability calibration training into text classification algorithm
Jing JIANG, Yu CHEN, Jieping SUN, Shenggen JU
Journal of Computer Applications    2022, 42 (6): 1789-1795.   DOI: 10.11772/j.issn.1001-9081.2021091638
Abstract252)   HTML7)    PDF (738KB)(56)       Save

The pre-training language models used for text representation have achieved high accuracy on various text classification tasks, but the following problems still remain: on the one hand, the category with the largest posterior probability is selected as the final classification result of the model after calculating the posterior probabilities on all categories in the pre-training language model. However, in many scenarios, the quality of the posterior probability itself can provide more reliable information than the final classification result. On the other hand, the classifier of the pre-training language model has performance degradation when assigning different labels to texts with similar semantics. In response to the above two problems, a model combining posterior probability calibration and negative example supervision named PosCal-negative was proposed. In PosCal-negative model, the difference between the predicted probability and the empirical posterior probability was dynamically penalized in an end-to-and way during the training process, and the texts with different labels were used to realize the negative supervision of the encoder, so that different feature vector representations were generated for different categories. Experimental results show that the classification accuracies of the proposed model on two Chinese maternal and child care text classification datasets MATINF-C-AGE and MATINF-C-TOPIC reach 91.55% and 69.19% respectively, which are 1.13 percentage points and 2.53 percentage points higher than those of Enhanced Representation through kNowledge IntEgration (ERNIE) model respectively.

Table and Figures | Reference | Related Articles | Metrics